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(54) A system and method for beamforming using a microphone array 



(57) The ability to combine multiple audio signals 
captured from the microphones in a microphone array 
is frequently used in beamforming systems. Typically, 
beamforming involves processing the output audio sig- 
nals of the microphone array in such a way as to make 
the microphone array act as a highly directional micro- 
phone. In other words, beamforming provides a "listen- 
ing beam" which points to a particular sound source 



while often filtering out other sounds. A "generic beam- 
former," as described herein automatically designs a set 
of beams (i.e., beamforming) that cover a desired angu- 
lar space range within a prescribed search area. Beam 
design is a function of microphone geometry and oper- 
ational characteristics, and also of noise models of the 
environment around the microphone array. One advan- 
tage of the generic beamf ormer is that it is applicable to 
any microphone array geometry and microphone type. 
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EP 1 571 875 A2 

Description 
BACKGROUND 
5 Technical Field: 

[0001] The invention is related to finding the direction to a sound source in a prescribed search area using a beam- 
steering approach with a microphone array, and in particular, to a system and method that provides automatic beam- 
forming design for any microphone array geometry and for any type of microphone. 

10 

Background Art: 

[0002] Localization of a sound source or direction within a prescribed region is an important element of many systems. 
For example, a number of conventional audio conferencing applications use microphone arrays with conventional 
is sound source localization (SSL) to enable speech or sound originating from a particular point or direction to be effec- 
tively isolated and processed as desired. 

[0003] For example, conventional microphone arrays typically include an arrangement of microphones in some pre- 
determined layout. These microphones are generally used to simultaneously capture sound waves from various direc- 
tions and originating from different points in space. Conventional techniques such as SSL are then used to process 

20 these signals for localizing the source of the sound waves and for reducing noise. One type of conventional SSL 
processing uses beamsteering techniques for finding the direction to a particular sound source. In other words, beam- 
steering techniques are used to combine the signals from all microphones in such a way as to make the microphone 
array act as a highly directional microphone, pointing a "listening beam" to the sound source. Sound capture is then 
attenuated for sounds coming from directions outside that beam. Such techniques allow the microphone array to sup- 

25 press a portion of ambient noises and reverberated waves (generated by reflections of sound on wails and objects in 
the room), and thus providing a higher signal to noise ratio (SNR) for sound signals originating from within the target 
beam. 

[0004] Beamsteering typically allows beams to be steered or targeted to provide sound capture within a desired 
spatial area or region, thereby improving the signal-to-noise ratio (SNR) of the sounds recorded from that region. 

30 Therefore, beamsteering plays an important role in spatial filtering, i.e., pointing a "beam" to the sound source and 
suppressing any noises coming from other directions. In some cases the direction to the sound source is used for 
speaker tracking and post-processing of recorded audio signals. In the context of a video conferencing system, speaker 
tracking is often used for dynamically directing a video camera toward the person speaking. 
[0005] In general, as is well known to those skilled in the art, beamsteering involves the use of beamforming tech- 

35 niques for forming a set of beams designed to cover particular angular regions within a prescribed area. A beamformer 
is basically a spatial filter that operates on the output of an array of sensors, such as microphones, in order to enhance 
the amplitude of a coherent wavefront relative to background noise and directional interference. A set of signal process- 
ing operators (usually linear fitters) is then applied to the signals form each sensor, and the outputs of those filters are 
combined to form beams, which are pointed, or steered, to reinforce inputs from particular angular regions and attenuate 

40 inputs from other angular regions. 

[0006] The "pointing direction" of the steered beam is often referred to as the maximum or main response angle 
(MRA), and can be arbitrarily chosen for the beams. In other words, beamforming techniques are used to process the 
input from multiple sensors to create a set of steerable beams having a narrow angular response area in a desired 
direction (the MRA). Consequently, when a sound is received from within a given beam, the direction of that sound is 

45 known (i.e., SSL), and sounds emanating from other beams may be filtered or otherwise processed, as desired. 

[0007] One class of conventional beamforming algorithms attempts to provide optimal noise suppression by finding 
parametric solutions for known microphone array geometries. Unfortunately, as a result of the high complexity, and 
thus large computational overhead, of such approaches, more emphasis has been given to finding near-optimal solu- 
tions, rather than optimal solutions. These approaches are often referred to as "fixed-beam formation." 

so [0008] In general, with fixed-beam formation, the beam shapes do not adapt to changes in the surrounding noises 
and sound source positions. Further, the near-optimal solutions offered by such approaches tend to provide only near- 
optimal noise suppression for off-beam sounds or noise. Consequently, there is typically room for improvement in noise 
or sound suppression offered by such conventional beamforming techniques. Finally, such beamforming algorithms 
tend to be specifically adapted for use with particular microphone arrays. Consequently, a beamforming technique 

55 designed for one particular microphone array may not provide acceptable results when applied to another microphone 
array of a different geometry. 

[0009] Other conventional beamforming techniques involve what is known as "adaptive beamforming." Such tech- 
niques are capable of providing noise suppression based on little or no a priori knowledge of the microphone array 
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geometry. Such algorithms adapt to changes in ambient or background noise and to the sound source position by 
attempting to converge upon an optimal solution as a function of time, thereby providing optimal noise suppression 
after convergence. Unfortunately, one disadvantage of such techniques is their significant computational requirements 
and slow adaptation, which makes them less robust to wide varieties in application scenarios. 
5 [0010] Consequently, what is needed is a system and method for providing better optimized beamforming solutions 
for microphone arrays. Further, such a system and method should reduce computational overhead so that real-time 
beamforming is realized. Finally, such a system and method should be applicable for microphone arrays of any geom- 
etry and including any type of microphone. 

10 SUMMARY 

[0011] The ability to combine multiple audio signals captured from the microphones in a microphone array is fre- 
quently used in beamforming systems. In general, beamforming operations are applicable to processing the signals 
of a number of receiving arrays, including microphone arrays, sonar arrays, directional radio antenna arrays, radar 

15 arrays, etc. For example, in the case of a microphone array, beamforming involves processing output audio signals of 
the microphone array in such a way as to make the microphone array act as a highly directional microphone. In other 
words, beamforming provides a "listening beam" which points to, and receives, a particular sound source while atten- 
uating other sounds and noise, including, for example, reflections, reverberations, interference, and sounds or noise 
coming from other directions or points outside the primary beam. Pointing of such beams is typically referred to as 

20 "beamsteering." 

[0012] Note that beamforming systems also frequently apply a number of types of noise reduction or other filtering 
or post-processing to the signal output of the beamf ormer. Further, time- or frequency-domain pre-processing of sensor 
array outputs prior to beamforming operations is also frequently used with conventional beamforming systems. How- 
ever, for purposes of explanation, the following discussion will focus on beamforming design for microphone arrays of 
25 arbitrary geometry and microphone type, and will consider only the noise reduction that is a natural consequence of 
the spatial filtering resulting from beamforming and beamsteering operations. Any desired conventional pre- or post- 
processing or filtering of the beamformer input or output should be understood to be within the scope of the description 
of the generic beamformer provided herein. 

[0013] A "generic beamformer," as described herein, automatically designs a set of beams (i.e., beamforming) that 
30 cover a desired angular space range. However, unlike conventional beamforming techniques, the generic beamformer 
described herein is capable of automatically adapting to any microphone array geometry, and to any type of microphone. 
Specifically, the generic beamformer automatically designs an optimized set of steerable beams for microphone arrays 
of arbitrary geometry and microphone type by determining optimal beam widths as a function of frequency to provide 
optimal signal-to-noise ratios for in-beam sound sources while providing optimal attenuation or filtering for ambient 
35 and off-beam noise sources. The generic beamformer provides this automatic beamforming design through a novel 
error minimization process that automatically determines optimal frequency-dependant beam widths given local noise 
conditions and microphone array operational characteristics. Note that while the generic beamformer is applicable to 
sensor arrays of various types, for purposes of explanation and clarity, the following discussion will assume that the 
sensor array is a microphone array comprising a number of microphones with some known geometry and microphone 
40 directivity. 

[0014] In general, the generic beamformer begins the design of optimal fixed beams for a microphone array by first 
computing a frequency-dependant "weight matrix" using parametric information describing the operational character- 
istics and geometry of the microphone array, in combination with one or more noise models that are automatically 
generated or computed for the environment around the microphone array. This weight matrix is then used for frequency 
45 domain weighting of the output of each microphone in the microphone array in frequency-domain beamfomriing process- 
ing of audio signals received by the microphone array. 

[0015] The weights computed for the weight matrix are determined by calculating frequency-domain weights for a 
desired "focus points" distributed throughout the workspace around the microphone array. The weights in this weight 
matrix are optimized so that beams designed by the generic beamformer will provide maximal noise suppression (based 
so on the computed noise models) under the constraints of unit gain and zero phase shift in any particular focus point for 
each frequency band. These constraints are applied for an angular area around the focus point, called the "focus 
width." This process is repeated for each frequency band of interest, thereby resulting in optimal beam widths that vary 
as a function of frequency for any given focus point. 

[0016] In one embodiment, beamforming processing is performed using a frequency-domain technique referred to 
55 as Modulated Complex Lapped Transforms (MCLT). However, while the concepts described herein use MCLT domain 
processing by way of example, it should be appreciated by those skilled in the art, that these concepts are easily 
adaptable to other frequency-domain decompositions, such as, for example, fast Fourier transform (FFT) or FFT-based 
filter banks. Note that because the weights are computed for frequency domain weighting, the weight matrix is an NX M 
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matrix, where N is the number of MCLT frequency bands (i.e., MCLT subbands) in each audio frame and M is the 
number of microphones in the array. Therefore, assuming, for example, the use of 320 frequency bins for MCLT com- 
putations, an optimal beam width for any particular focus point can be described by plotting gain as a function of 
incidence angle and frequency for each of the 320 MCLT frequency coefficients. Note that using a large number of 
5 MCLT subbands (e.g. 320) allows for two important advantages of the frequency-domain technique: i) fine tuning of 
the beam shapes for each frequency subband; and ii) simplifying the filter coefficients for each subband to single 
complex-valued gain factors, allowing for computationally efficient implementations. 

[0017] The parametric information used for computing the weight matrix includes the number of microphones in the 
array, the geometric layout of the microphones in the array, and the directivity pattern of each microphone in the array. 

10 The noise models generated for use in computing the weight matrix distinguish at least three types of noise, including 
isotropic ambient noise (i.e., background noise such as "white noise" or other relatively uniformly distributed noise), 
instrumental noise (i.e., noise resulting from electrical activity within the electrical circuitry of the microphone array and 
array connection to an external computing device or other external electrical device) and point noise sources (such 
as, for example, computer fans, traffic noise through an open window, speakers that should be suppressed, etc.) 

is [0018] Therefore, given the aforementioned noise models, the solution to the problem of designing optimal fixed 
beams for the microphone array is similar to a typical minimization problem with constraints that is solved by using 
methods for mathematical multidimensional optimization (simplex, gradient, etc.). However, given the relatively high 
dimensionality of the weight matrix (2M real numbers per frequency band, for a total of NX2M numbers), which can 
be considered as a multimodal hypersurface, and because the functions are nonlinear, finding the optimal weights as 

20 points in the multimodal hypersurface is very computationally expensive, as it typically requires multiple checks for 
local minima. 

[0019] Consequently, in one embodiment, rather than directly finding optimal points in this multimodal hypersurface, 
the generic beamforrner first substitutes direct multidimensional optimization for computation of the weight matrix with 
an error minimizing pattern synthesis, followed by a single dimensional search towards an optimal beam focus width 
25 for each frequency band. Any conventional error minimization technique can be used here, such as, for example, least- 
squares or minimum mean-square error (MMSE) computations, minimum absolute error computations, min-max error 
computations, equiripple solutions, etc. 

[0020] In general, in finding the optimal solution for the weight matrix, two contradicting effects are balanced. Spe- 
cifically, given a narrow focus area for the beam shape, ambient noise energy will naturally decrease due to increased 
30 directivity. In addition, non-correlated noise (including electrical circuit noise) will naturally increase since a solution for 
better directivity will consider smaller and smaller phase differences between the output signals from the microphones, 
thereby boosting the non-correlated noise. Conversely, when the target focus area of the beam shape is larger, there 
will naturally be more ambient noise energy, but less non-correlated noise energy. 

[0021] Therefore, the generic beamforrner considers a balance of the above-noted factors in computing a minimum 
35 error for a particular focus area width to identify the optimal solution for weighting each MCLT frequency band for each 

microphone in the array. This optimal solution is then determined through pattern synthesis which identifies weights 

that meet the least squares (or other error minimization technique) requirement for particular target beam shapes. 

Fortunately, by addressing the problem in this manner, it can be solved using a numerical solution of a linear system 

of equations, which is significantly faster than multidimensional optimization. Note that because this optimization is 
40 computed based on the geometry and directivity of each individual microphone in the array, optimal beam design will 

vary, even within each specific frequency band, as a function of a target focus point for any given beam around the 

microphone array. 

[0022] Specifically, the beamforrner design process first defines a set of "target beam shapes" as a function of some 
desired target beam width focus area (i.e. , 2-degrees, 5-degrees, 1 0-degrees, etc.). In general, any conventional func- 

45 tion which has a maximum of one and decays to zero can be used to define the target beam shape, such as, for 
example, rectangular functions, spline functions, cosine functions, etc. However, abrupt functions such as rectangular 
functions can cause ripples in the beam shape. Consequently, better results are typically achieved using functions 
which smoothly decay from one to zero, such as, for example, cosine functions. However, any desired function may 
be used here in view of the aforementioned constraints of a decay function (linear or non-linear) from one to zero, or 

so some decay function which is weighted to force levels from one to zero. 

[0023] Given the target beam shapes, a "target weight function" is then defined to address whether each target or 
focus point is in, out, or within a transition area of a particular target beam shape. Typically a transition area of about 
one to three times the target beam width has been observed to provide good results; however, the optimal size of the 
transition area is actually dependent upon the types of sensors in the array, and on the environment of the workspace 

55 around the sensor array. Note that the focus points are simply a number of points (preferably larger than the number 
of microphones) that are equally spread throughout the workspace around the array (i.e., using an equal circular spread 
for a circular array, or an equal arcing spread for a linear array). The target weight functions then provide a gain for 
weighting each target point depending upon where those points are relative to a particular target beam. 
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[0024] The purpose of providing the target weight functions is to minimize the effects of signals originating from points 
outside the main beam on beamformer computations. Therefore, in a tested embodiment, target points Inside the target 
beam were assigned a gain of 1 .0 (unit gain); target points within the transition area were assigned a gain of 0.1 to 
minimize the effect of such points on beamf orming computations while still considering their effect; finally points outside 
5 of the transition area of the target beam were assigned a gain of 2.0 so as to more fully consider and strongly reduce 
the amplitudes of sideiobes on the final designed beams. Note that using too high of a gain for target points outside 
of the transition area can have the effect of overwhelming the effect of target points within the target beam, thereby 
resulting in less than optimal beamforming computations. 

[0025] Next, given the target beam shape and target weight functions, the next step is to compute a set of weights 
10 that will fit real beam shapes (using the known directivity patterns of each microphone in the array as the real beam 
shapes) into the target beam shape for each target point by using an error minimization technique to minimize the total 
noise energy for each MCLT frequency subband for each target beam shape. The solution to this computation is a set 
of weights that match a real beam shape to the target beam shape. However, this set of weights does not necessarily 
meet the aforementioned constraints of unit gain and zero phase shift in the focus point for each work frequency band. 
15 In other words, the initial set of weights may provide more or less than unit gain for a sound source within the beam. 
Therefore, the computed weights are normalized such that there is a unit gain and a zero phase shift for any signals 
originating from the focus point. 

[0026] At this point, the generic beamformer has not yet considered an overall minimization of the total noise energy 
as a function of beam width. Therefore, rather than simply computing the weights for one desired target beam width, 

20 as described above, normalized weights are computed for a range of target beam widths, ranging from some prede- 
termined minimum to some predetermined maximum desired angle. The beam width step size can be as small or as 
large as desired (i.e., step sizes of 0.5, 1 , 2, 5, 1 0 degrees, or any other step size, may be used, as desired). A one- 
dimensional optimization is then used to identify the optimum beam width for each frequency band. Any of a number 
of well-known nonlinear function optimization techniques can be employed, such a gradient descent methods, search 

25 methods, etc. In other words, the total noise energy is computed for each target beam width throughout some range 
of target beam widths using any desired angular step size. These total noise energies are then simply compared to 
identify the beam width at each frequency exhibiting the lowest total noise energy for that frequency. The end result is 
an optimized beam width that varies as a function of frequency for each target point around the sensor array. 
[0027] Note that in one embodiment, this total lowest noise energy is considered as a function of particular frequency 

30 ranges, rather than assuming that noise should be attenuated equally across all frequency ranges. In particular, in 
some cases, it is desirable to minimize the total noise energy within only certain frequency ranges, or to more heavily 
attenuate noise within particular frequency ranges. In such cases, those particular frequency ranges are given more 
consideration in identifying the target beam width having the lowest noise energy. One way of determining whether 
noise is more prominent in any particular frequency range is to simply perform a conventional frequency analysis to 

35 determine noise energy levels for particular frequency ranges. Frequency ranges with particularly high noise energy 
levels are then weighted more heavily to increase their effect on the overall beamforming computations, thereby re- 
sulting in a greater attenuation of noise within such frequency ranges. 

[0028] The normalized weights for the beam width having the lowest total noise energy at each frequency level are 
then provided for the aforementioned weight matrix. The workspace is then divided into a number of angular regions 
40 corresponding to the optimal beam width for any given frequency with respect to the target point at which the beam is 
being directed. Note that beams are directed using conventional techniques, such as, for example sound source lo- 
calization (SSL). Direction of such beams to particular points around the array is a concept well known to those skilled 
in the art, and will not be described in detail herein. 

[0029] Further, it should be noted that particular applications may require some degree of beam overlap to provide 
45 for improved signal source localization. In such cases, the amount of desired overlap between beams is simply used 
to determine the number of beams needed to provide full coverage of the desired workspace. One example of an 
application wherein beam overlap is used is provided in a copending patent application entitled "A SYSTEM AND 
METHOD FOR IMPROVING THE PRECISION OF LOCALIZATION ESTIMATES; filed March 1 , 2004, and assigned 
Serial Number 10/791,252 the subject matter of which is incorporated herein by this reference. Thus, for example, 
so where a 50-percent beam overlap is desired, the number of beams will be doubled, and using the aforementioned 
example of the 20-degree beam width at a particular frequency for a circular workspace, the workspace would be 
divided into 36 overlapping 20-degree beams, rather than using only 18 beams. 

[0030] In a further embodiment, the beamforming process may evolve as a function of time. In particular, as noted 
above, the weight matrix and optimal beam widths are computed, in part, based on the noise models computed for the 
55 workspace around the microphone array. However, it should be clear that noise levels and sources often change as a 
function of time. Therefore, in one embodiment, noise modeling of the workspace environment is performed either 
continuously, or at regular or user specified intervals. Given the new noise models, the beamforming design processes 
described above are then used to automatically update the set of optimal beams for the workspace. 
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[0031] In view of the above summary, it is clear that the generic beamformer described herein provides a system 
and method for designing an optimal beam set for microphone arrays of arbitrary geometry and microphone type. In 
addition to the just described benefits, other advantages of this system and method will become apparent from the 
detailed description which follows hereinafter when taken in conjunction with the accompanying drawing figures. 

5 

DESCRIPTION OF THE DRAWINGS 

[0032] The specific features, aspects, and advantages of the present invention will become better understood with 
regard to the following description, appended claims, and accompanying drawings where: 

10 

FIG. 1 is a general system diagram depicting a general-purpose computing device constituting an exemplary sys- 
tem for implementing a generic beamformer for designing an optimal beam set for microphone arrays of arbitrary 
geometry and microphone type. 

15 FIG. 2 illustrates an exemplary system diagram showing exemplary program modules for implementing a generic 

beamformer for designing optimal beam sets for microphone arrays of arbitrary geometry and microphone type. 

FIG. 3 is a general flowgraph illustrating MCLT-based processing of input signals for a beam computed by the 
generic beamformer of FIG. 2 to provide an output audio signal for a particular target point. 

20 

FIG. 4 provides an example of the spatial selectivity (gain) of a beam generated by the generic beamformer of 
FIG. 2, as a function of frequency and beam angle. 

FIG. 5 provides an exemplary operational flow diagram illustrating the operation of a generic beamformer for de- 
25 signing optimal beams for a microphone array. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0033] In the following description of the preferred embodiments of the present invention, reference is made to the 
30 accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific embodiments 
in which the invention may be practiced. It is understood that other embodiments may be utilized and structural changes 
may be made without departing from the scope of the present invention. 

1 .0 Exemplary Operating Environment: 

35 

[0034] Figure 1 illustrates an example of a suitable computing system environment 1 00 with which the invention may 
be implemented. The computing system environment 100 is only one example of a suitable computing environment 
and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the 
computing environment 1 00 be interpreted as having any dependency or requirement relating to any one orcombination 

40 of components illustrated in the exemplary operating environment 1 00. 

[0035] The invention is operational with numerous other general purpose or special purpose computing system en- 
vironments or configurations. Examples of well known computing systems, environments, and/or configurations that 
may be suitable for use with the invention include, but are not limited to, personal computers, server computers, hand- 
held, laptop or mobile computer or communications devices such as cell phones and PDA's, multiprocessor systems, 

45 microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, 
mainframe computers, distributed computing environments that include any of the above systems or devices, and the 
like. 

[0036] The invention may be described in the general context of computer-executable instructions, such as program 
modules, being executed by a computer in combination with hardware modules, including components of a microphone 

so array 1 98, or other receiver array (not shown), such as, for example, a directional radio antenna array, a radar receiver 
array, etc. Generally, program modules include routines, programs, objects, components, data structures, etc., that 
perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed 
computing environments where tasks are performed by remote processing devices that are linked through a commu- 
nications network. In a distributed computing environment, program modules may be located in both local and remote 

55 computer storage media Including memory storage devices. With reference to Figure 1, an exemplary system for 
implementing the invention includes a general-purpose computing device in the form of a computer 110. 
[0037] Components of computer 110 may include, but are not limited to, a processing unit 120, a system memory 
130, and a system bus 121 that couples various system components including the system memory to the processing 
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unit 120. The system bus 121 may be any of several types of bus structures including a memory bus or memory 
controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not 
limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, 
Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component 

s Interconnect (PCI) bus also known as Mezzanine bus. 

[0038] Computer 1 1 0 typically includes a variety of computer readable media. Computer readable media can be any 
available media that can be accessed by computer 110 and includes both volatile and nonvolatile media, removable 
and non-removable media. By way of example, and not limitation, computer readable media may comprise computer 
storage media and communication media. Computer storage media includes volatile and nonvolatile removable and 

io non-removable media implemented in any method or technology for storage of information such as computer readable 
instructions, data structures, program modules, or other data. 

[0039] Computer storage media includes, but is not limitedto, RAM, ROM, PROM, EPROM, EEPROM, flash memory, 
or other memory technology; CD-ROM, digital versatile disks (DVD), or other optical disk storage; magnetic cassettes, 
magnetic tape, magnetic disk storage, or other magnetic storage devices; or any other medium which can be used to 

15 store the desired information and which can be accessed by computer 1 1 0. Communication media typically embodies 
computer readable instructions, data structures, program modules or other data in a modulated data signal such els a 
carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data 
signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode 
information in the signal. By way of example, and not limitation, communication media includes wired media such as 

20 a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless 
media. Combinations of any of the above should also be included within the scope of computer readable media. 
[0040] The system memory 1 30 includes computer storage media in the form of volatile and/or nonvolatile memory 
such as read only memory (ROM) 131 and random access memory (RAM) 132. A basic input/output system 133 
(BIOS), containing the basic routines that help to transfer information between elements within computer 110, such as 

25 during start-up, is typically stored in ROM 131. RAM 132 typically contains data and/or program modules that are 
immediately accessible to and/or presently being operated on by processing unit 120. By way of example, and not 
limitation, Figure 1 illustrates operating system 134, application programs 135, other program modules 136, and pro- 
gram data 137. 

[0041] The computer 110 may also include other removable/non-removable, volatile/nonvolatile computer storage 
30 media. By way of example only, Figure 1 illustrates a hard disk drive 141 that reads from or writes to non-removable, 
nonvolatile magnetic media, a magnetic disk drive 151 that reads from or writes to a removable, nonvolatile magnetic 
disk 152, and an optical disk drive 155 that reads from or writes to a removable, nonvolatile optical disk 156 such as 
a CD ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media that 
can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash 
35 memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The hard disk 
drive 141 is typically connected to the system bus 121 through a non-removable memory interface such as interface 
140, and magnetic disk drive 151 and optical disk drive 155 are typically connected to the system bus 121 by a remov- 
able memory interface, such as interface 150. 

[0042] The drives and their associated computer storage media discussed above and illustrated in Figure 1 , provide 
^0 storage of computer readable instructions, data structures, program modules and other data for the computer 1 1 0. In 
Figure 1, for example, hard disk drive 141 is illustrated as storing operating system 144, application programs 145, 
other program modules 146, and program data 147. Note that these components can either be the same as or different 
from operating system 134, application programs 135, other program modules 136, and program data 137. Operating 
system 144, application programs 145, other program modules 146, and program data 147 are given different numbers 
4 $ here to illustrate that, at a minimum, they are different copies. A user may enter commands and information into the 
computer 110 through input devices such as a keyboard 162 and pointing device 161, commonly referred to as a 
mouse, trackball, or touch pad. 

[0043] Other input devices (not shown) may include a joystick, game pad, satellite dish, scanner, radio receiver, and 
a television or broadcast video receiver, or the like. Still further input devices (not shown) may include receiving arrays 

so or signal input devices, such as, for example, a directional radio antenna array, a radar receiver array, etc. These and 
other input devices are often connected to the processing unit 120 through a wired or wireless user input interface 1 60 
that is coupled to the system bus 121 , but may be connected by other conventional interface and bus structures, such 
as, for example, a parallel port, a game port, a universal serial bus (USB), an IEEE 1394 interface, a Bluetooth™ 
wireless interface, an IEEE 802.11 wireless interface, etc. Further, the computer 110 may also include a speech or 

55 audio input device, such as a microphone or a microphone array 198, as well as a loudspeaker 197 or other sound 
output device connected via an audio interface 1 99, again including conventional wired or wireless interfaces, such 
as, for example, parallel, serial, USB, IEEE 1394, Bluetooth™, etc. 

[0044] A monitor 1 91 or other type of display device is also connected to the system bus 1 21 via an interface, such 
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as a video interface 190. in addition to the monitor, computers may also include other peripheral output devices such 
as a printer 196, which may be connected through an output peripheral interface 195. 

[0045] The computer 1 1 0 may operate in a networked environment using logical connections to one or more remote 
computers, such as a remote computer 1 80. The remote computer 1 80 may be a personal computer, a server, a router, 

5 a network PC, a peer device, or other common network node, and typically includes many or all of the elements de- 
scribed above relative to the computer 1 1 0, although only a memory storage device 1 81 has been illustrated in Figure 
1 . The logical connections depicted In Figure 1 include a local area network (LAN) 171 and a wide area network (WAN) 
1 73, but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide 
computer networks, intranets, and the Internet. 

10 [0046] When used in a LAN networking environment, the computer 110 is connected to the LAN 171 through a 
network interface or adapter 170. When used in a WAN networking environment, the computer 110 typically includes 
a modem 172 or other means for establishing communications over the WAN 173, such as the Internet. The modem 
172, which may be internal or external, may be connected to the system bus 121 via the user input interface 1 60, or 
other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 110, 

is or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation, Figure 
1 illustrates remote application programs 1 85 as residing on memory device 1 81 . It will be appreciated that the network 
connections shown are exemplary and other means of establishing a communications link between the computers 
may be used. 

[0047] The exemplary operating environment having now been discussed, the remaining part of this description will 
20 be devoted to a discussion of a system and method for automatically designing optimal beams for microphones of 
arbitrary geometry and microphone type. 

2.0 Introduction: 

25 [0048] A "generic beamformer," as described herein, automatically designs a set of beams (i.e., beamforming) that 
cover a desired angular space range or "workspace." Such beams may then be used to localize particular signal sources 
within a prescribed search area within the workspace around a sensor array. For example, typical space ranges may 
include a 360-degree range for a circular microphone array in a conference room, or an angular range of about 120- 
to 150-degrees for a linear microphone array as is sometimes employed for personal use with a desktop or PC-type 

30 computer. 

[0049] However, unlike conventional beamforming techniques, the generic beamformer described herein is capable 
of designing a set of optimized beams for any sensor array given geometry and sensor characteristics. For example, 
in the case of a microphone array, the geometry would be the number and position of microphones in the array, and 
the characteristics would include microphone directivity for each microphone in the array. 

35 [0050] Specifically, the generic beamformer designs an optimized set of steerabte beams for sensor arrays of arbitrary 
geometry and sensor type by determining optimal beam widths as a function of frequency to provide optimal signal- 
to-noise ratios for in-beam sound sources while providing optimal attenuation or filtering for ambient and off-beam 
noise sources. The generic beamformer provides this beamforming design through a novel error minimization process 
that determines optimal frequency-dependant beam widths given local noise conditions and microphone array opera- 

40 tionai characteristics. Note that while the generic beamformer is applicable to sensor arrays of various types, for pur- 
poses of explanation and clarity, the following discussion will assume that the sensor array is a microphone array 
comprising a number of microphones with some known geometry and microphone directivity. 
[0051] Note that beamforming systems also frequently apply a number of types of noise reduction or other filtering 
or post-processing to the signal output of the beamformer. Further, time- or frequency-domain pre-processing of sensor 

45 array inputs prior to beamforming operations is also frequently used with conventional beamforming systems. However, 
for purposes of explanation, the following discussion will focus on beamforming design for microphone arrays of arbi- 
trary geometry and microphone type, and will consider only the noise reduction that is a natural consequence of the 
spatial filtering resulting from beamforming and beamsteering operations. Any desired conventional pre- or post- 
processing or filtering of the beamformer input or output should be understood to be within the scope of the description 

50 of the generic beamformer provided herein. 

[0052] Further, unlike conventional fixed-beam formation and adaptive beamforming techniques which typically op- 
erate in a time-domain, the generic beamformer provides all beamforming operations in the frequency domain. Most 
conventional audio processing, including, for example, filtering, spectral analysis, audio compression, signature ex- 
traction, etc., typically operate in a frequency domain using Fast Fourier Transforms (FFT), or the like. Consequently, 

55 conventional beamforming systems often first provide beamforming operations in the time domain, and then convert 
those signals to a frequency domain for further processing, and then, finally, covert those signals back to a time-domain 
signal for playback. 

[0053] Therefore, one advantage of the generic beamformer described herein is that unlike most conventional beam- 
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forming techniques, it provides beamforming processing entirely within the frequency domain. Further, in one embod- 
iment, this frequency domain beamforming processing is performed using a frequency-domain technique referred to 
as Modulated Complex Lapped Transforms (MCLT), because MCLT-domain processing has some advantages with 
respect to integration with other audio processing modules, such as compression and decompression modules (co- 
s decs). 

[0054] However, while the concepts described herein use MCLT domain processing by way of example, it should be 
appreciated that these concepts are easily adaptable to other frequency-domain decompositions, such as, for example, 
FFT or FFT-based filter banks. Consequently, signal processing, such as additional filtering, generating of digital audio 
signatures, audio compression, etc., can be performed directly in the frequency domain directly from the beamformer 
io output without first performing beamforming processing in the time-domain and then converting to the frequency do- 
main. In addition, the design of the generic beamformer guarantees linear processing and absence of non-linear dis- 
tortions in the output signal thereby further reducing computational overhead and signal distortions. 

2.1 System Overview: 

15 

[0055] In general, the generic beamformer begins the design of optimal fixed beams for a microphone array by first 
computing a frequency-dependant "weight matrix" using parametric information describing the operational character- 
istics and geometry of the microphone array, in combination with one or more noise models that are automatically 
generated or computed for the environment around the microphone array. This weight matrix is then used for frequency 
20 domain weighting of the output of each microphone In the microphone array in frequency-domain beamforming process- 
ing of audio signals received by the microphone array. 

[0056] The weights computed for the weight matrix are determined by calculating frequency-domain weights for a 
desired "focus points" distributed throughout the workspace around the microphone array. The weights in this weight 
matrix are optimized so that beams designed by the generic beamformer will provide maximal noise suppression (based 
25 on the computed noise models) under the constraints of unit gain and zero phase shift in any particular focus point for 
each frequency band. These constraints are applied for an angular area around the focus point, called the "focus 
width." This process is repeated for each frequency band of interest, thereby resulting in optimal beam widths that vary 
as a function of frequency for any given focus point. 

[0057] In one embodiment, beamforming processing is performed using a frequency-domain technique referred to 
30 as Modulated Complex Lapped Transforms (MCLT). However, while the concepts described herein use MCLT domain 
processing by way of example, it should be appreciated by those skilled in the art, that these concepts are easily 
adaptable to other frequency-domain decompositions, such as, for example, FFT or FFT-based filter banks. Note that 
because the weights are computed for frequency domain weighting, the weight matrix is an NxM matrix, where N is 
the number of MCLT frequency bands (i.e., MCLT subbands) in each audio frame and M is the number of microphones 
35 in the array. Therefore, assuming, for example, the use of 320 frequency bins for MCLT computations, an optimal beam 
width for any particular focus point can be described by plotting gain as a function of Incidence angle and frequency 
for each of the 320 MCLT frequency coefficients. 

[0058] Further, it should be noted that when using MCLT processing for beamforming operations, using a larger 
number of MCLT subbands (e.g., 320 subbands, as in the preceding example) provides two important advantages of 
40 this frequency-domain technique: i) fine tuning of the beam shapes for each frequency subband; and ii) simplifying the 
filter coefficients for each subband to single complex-valued gain factors, allowing for computationally efficient imple- 
mentations. 

[0059] The parametric information used for computing the weight matrix includes the number of microphones in the 
array, the geometric layout of the microphones in the array, and the directivity pattern of each microphone in the array. 

45 The noise models generated for use in computing the weight matrix distinguish at least three types of noise, including 
isotropic ambient noise (i.e., background noise such as "white noise" or other relatively uniformly distributed noise), 
instrumental noise (i.e., noise resulting from electrical activity within the electrical circuitry of the microphone array and 
array connection to an external computing device or other external electrical device) and point noise sources (such 
as, for example, computer fans, traffic noise through an open window, speakers that should be suppressed, etc.) 

so [0060] Therefore, given the aforementioned noise models, the solution to the problem of designing optimal fixed 
beams for the microphone array is similar to a typical minimization problem with constraints that is solved by using 
methods for mathematical multidimensional optimization (simplex, gradient, etc.). However, given the relatively high 
dimensionality of the weight matrix (2M real numbers per frequency band, for a total of Nx2M numbers), which can 
be considered as a multimodal hypersurface, and because the functions are nonlinear, finding the optimal weights as 

55 points in the multimodal hypersurface is very computationally expensive, as it typically requires multiple checks for 
local minima. 

[0061] Consequently, in one embodiment, rather than directly finding optimal points in this multimodal hypersurface, 
the generic beamformer first substitutes direct multidimensional optimization for computation of the weight matrix with 
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an error minimizing pattern synthesis, followed by a single dimensional search towards an optimal beam focus width. 
Any conventional error minimization technique can be used here, such as, for example, least-squares or minimum 
mean-square error (MMSE) computations, minimum absolute error computations, min-max error computations, equirip- 
pie solutions, etc. 

s [0062] In general, in finding the optimal solution for the weight matrix, two contradicting effects are balanced. Spe- 
cifically, given a narrow focus area for the beam shape, ambient noise energy will naturally decrease due to increased 
directivity. In addition, non-correlated noise (including electrical circuit noise) will naturally increase since a solution for 
better directivity will consider smaller and smaller phase differences between the output signals from the microphones, 
thereby boosting the non-correlated noise. Conversely, when the target focus area of the beam shape is larger, there 

10 will naturally be more ambient noise energy, but less non-correlated noise energy. 

[0063] Therefore, the generic beamformer considers a balance of the above-noted factors in computing a minimum 
error for a particular focus area width to identify the optimal solution for weighting each MCLT frequency band for each 
microphone in the array. This optimal solution is then determined through pattern synthesis which identifies weights 
that meet the least squares (or other error minimization technique) requirement for particular target beam shapes. 

is Fortunately, by addressing the problem in this manner, it can be solved using a numerical solution of a linear system 
of equations, which is significantly faster than multidimensional optimization. Note that because this optimization is 
computed based on the geometry and directivity of each individual microphone in the array, optimal beam design will 
vary, even within each specific frequency band, as a function of a target focus point for any given beam around the 
microphone array. 

20 [0064] Specifically, the beamformer design process first defines a set of "target beam shapes" as a function of some 
desired target beam width focus area (i.e. , 2-degrees, 5-degrees, 1 0-degrees, etc.). In general, any conventional func- 
tion which has a maximum of one and decays to zero can be used to define the target beam shape, such as, for 
example, rectangular functions, spline functions, cosine functions, etc. However, abrupt functions such as rectangular 
functions can cause ripples in the beam shape. Consequently, better results are typically achieved using functions 

25 which smoothly decay from one to zero, such as, for example, cosine functions. However, any desired function may 
be used here in view of the aforementioned constraints of a decay function (linear or non-linear) from one to zero, or 
some decay function which is weighted to force levels from one to zero. 

[0065] Given the target beam shapes, a "target weight function" is then defined to address whether each target or 
focus point is in, out, or within a transition area of a particular target beam shape. Typically a transition area of about 

30 one to three times the target beam width has been observed to provide good results; however, the optimal size of the 
transition area is actually dependent upon the types of sensors in the array, and on the environment of the workspace 
around the sensor array. Note that the focus points are simply a number of points (preferably larger than the number 
of microphones) that are equally spread throughout the workspace around the array (i.e., using an equal circular spread 
for a circular array, or an equal arcing spread for a linear array). The target weight functions then provide a gain for 

35 weighting each target point depending upon where those points are relative to a particular target beam. 

[0066] The purpose of providing the target weight functions is to minimize the effects of signals originating from points 
outside the main beam on beamformer computations. Therefore, in a tested embodiment, target points inside the target 
beam were assigned a gain of 1 .0 (unit gain); target points within the transition area were assigned a gain of 0.1 to 
minimize the effect of such points on beamforming computations while still considering their effect; finally points outside 

40 of the transition area of the target beam were assigned a gain of 2.0 so as to more fully consider and strongly reduce 
the amplitudes of sidelobes on the final designed beams. Note that using too high of a gain for target points outside 
of the transition area can have the effect of overwhelming the effect of target points within the target beam, thereby 
resulting in less than optimal beamforming computations. 

[0067] Next, given the target beam shape and target weight functions, the next step is to compute a set of weights 
45 that will fit real beam shapes (using the known directivity patterns of each microphone in the array as the real beam 
shapes) into the target beam shape for each target point by using an error minimization technique to minimize the total 
noise energy for each MCLT frequency subband for each target beam shape. The solution to this computation is a set 
of weights that match a real beam shape to the target beam shape. However, this set of weights does not necessarily 
meet the aforementioned constraints of unit gain and zero phase shift in the focus point for each work frequency band. 
so in other words, the initial set of weights may provide more or less than unit gain for a sound source within the beam. 
Therefore, the computed weights are normalized such that there is a unit gain and a zero phase shift for any signals 
originating from the focus point. 

[0068] At this point, the generic beamformer has not yet considered an overall minimization of the total noise energy 
as a function of beam width. Therefore, rather than simply computing the weights for one desired target beam width, 
55 as described above, normalized weights are computed for a range of target beam widths, ranging from some prede- 
termined minimum to some predetermined maximum desired angle. The beam width step size can be as small or as 
large as desired (i.e., step sizes of 0.5, 1,2,5, 10 degrees, or any other step size, may be used, as desired). 
[0069] A one-dimensional optimization is then used to identify the optimum beam width for each frequency band. 
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Any of a number of well-known nonlinear function optimization techniques can be employed, such a gradient descent 
methods, search methods, etc. In other words, the total noise energy is computed for each target beam width throughout 
some range of target beam widths using any desired angular step size. These total noise energies are then simply 
compared to identify the beam width at each frequency exhibiting the lowest total noise energy for that frequency. The 

5 end result is an optimized beam width that varies as a function of frequency for each target point around the sensor array. 
[0070] Note that in one embodiment, this total lowest noise energy is considered as a function of particular frequency 
ranges, rather than assuming that noise should be attenuated equally across all frequency ranges. In particular, in 
some cases, it is desirable to minimize the total noise energy within only certain frequency ranges, or to more heavily 
attenuate noise within particular frequency ranges. In such cases, those particular frequency ranges are given more 

10 consideration in identifying the target beam width having the lowest noise energy. One way of determining whether 
noise is more prominent in any particular frequency range is to simply perform a conventional frequency analysis to 
determine noise energy levels for particular frequency ranges. Frequency ranges with particularly high noise energy 
levels are then weighted more heavily to increase their effect on the overall beamforming computations, thereby re- 
sulting in a greater attenuation of noise within such frequency ranges. 

is [0071] The normalized weights for the beam width having the lowest total noise energy at each frequency level are 
then provided for the aforementioned weight matrix. The workspace is then divided into a number of angular regions 
corresponding to the optimal beam width for any given frequency with respect to the target point at which the beam is 
being directed. Note that beams are directed using conventional techniques, such as, for example sound source lo- 
calization (SSL). Direction of such beams to particular points around the array is a concept well known to those skilled 

20 in the art, and will not be described in detail herein. 

[0072] Further, it should be noted that particular applications may require some degree of beam overlap to provide 
for improved signal source localization. In such cases, the amount of desired overlap between beams is simply used 
to determine the number of beams needed to provide full coverage of the desired workspace. One example of an 
application wherein beam overlap is used is provided in a copending patent application entitled "A SYSTEM AND 

25 METHOD FOR IMPROVING THE PRECISION OF LOCALIZATION ESTIMATES," filed March 1 , 2004, and assigned 
Serial Number 10/791,252, the subject matter of which is incorporated herein by this reference. Thus, for example, 
where a 50-percent beam overlap is desired, the number of beams will be doubled, and using the example of the 
20-degree beam width provided above for a circular workspace, the workspace would be divided into 36 overlapping 
20-degree beams, rather than using only 18 beams. 

30 [0073] In a further embodiment of the generic beamformer, the beamforming process may evolve as a function of 
time. In particular, as noted above, the weight matrix and optimal beam widths are computed, in part, based on the 
noise models computed for the workspace around the microphone array. However, it should be clear that noise levels 
and sources often change as a function of time. Therefore, in one embodiment, noise modeling of the workspace 
environment is performed either continuously, or at regular or user specified intervals. Given the new noise models, 

35 the beamforming design processes described above are then used to automatically define a new set of optimal beams 
for the workspace. 

[0074] Note that in one embodiment, the generic beamformer operates as a computer process entirely within a 
microphone array, with the microphone array itself receiving raw audio inputs from its various microphones, and then 
providing processed audio outputs. In this embodiment, the microphone array includes in integral computer processor 
40 which provides for the beamforming processing techniques described herein. However, microphone arrays with integral 
computer processing capabilities tend to be significantly more expensive than would be the case if the computer 
processing capabilities could be external to the microphone array, so that the microphone array only included micro- 
phones, preamplifiers, A/D converters, and some means of connectivity to an external computing device, such as, for 
example, a PC-type computer. 

45 [0075] Therefore, to address this issue, in one embodiment, the microphone array simply contains sufficient com- 
ponents to receive audio signals from each microphone array and provide those signals to an external computing 
device which then performs the beamforming processes described herein. In this embodiment, device drivers or device 
description files which contain data defining the operational characteristics of the microphone array, such as gain, 
sensitivity, array geometry, etc., are separately provided for the microphone array, so that the generic beamformer 

so residing within the external computing device can automatically design a set of beams that are automatically optimized 
for that specific microphone array in accordance with the system and method described herein. 
[0076] In a closely related embodiment, the microphone array includes a mechanism for automatically reporting its 
configuration and operational parameters to an external computing device. In particular, in this embodiment, the mi- 
crophone array includes a computer readable file or table residing in a microphone array memory, such as, for example 

55 a ROM, PROM, EPROM, EEPROM, or other conventional memory, which contains a microphone array device de- 
scription. This device description includes parametric information which defines operational characteristics and con- 
figuration of the microphone array. 

[0077] In this embodiment, once connected to the external computing device, the microphone array provides its 
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device description to the external computing device, which then uses the generic beamformer to automatically generate 
a set of beams automatically optimized for the connected microphone array. Further, the generic beamformer operating 
within the external computing device then performs all beamforming operations outside of the microphone array. This 
mechanism for automatically reporting the microphone array configuration and operational parameters to an external 
s computing device is described in detail in acopending patent application entitled "SELF-DESCRIPTIVE MICROPHONE 
ARRAY," filed February 9, 2004, and assigned Serial Number 10/775,371 , the subject matter of which is incorporated 
herein by this reference. 

[0078] In yet another related embodiment, the microphone array is provided with an integral self-calibration system 
that automatically determines frequency-domain responses of each preamplifier in the microphone array, and then 

10 computes frequency-domain compensation gains, so that the generic beamformer can use those compensation gains 
for matching the output of each preamplifier. As a result, there is no need to predetermine exact operational charac- 
teristics of each channel of the microphone array, or to use expensive matched electronic components. 
[0079] In particular, in this embodiment, the integral self-calibration system injects excitation pulses of a known mag- 
nitude and phase to all preamplifier inputs within the microphone array. The resulting analog waveform from each 

is preamplifier output is then measured. A frequency analysis, such as, for example, a Fast Fourier Transform (FFT), or 
other conventional frequency analysis, of each of the resulting waveforms is then performed. The results of this fre- 
quency analysis are then used to compute frequency-domain compensation gains for each preamplifier for matching 
or balancing the responses of ail of the preamplifiers with each other. This integral self-calibration system is described 
in detail in a copending patent application entitled "AN ALOG PREAMPLIFIER MEASUREMENT FOR A MICROPHONE 

20 ARRAY," filed February 4, 2004, and assigned Serial Number 10/772,528, the subject matter of which is incorporated 
herein by this reference. 

22 System Architecture: 

25 [0080] The processes summarized above are illustrated by the general system diagram of FIG. 2. In particular, the 
system diagram of FIG. 2 illustrates the interrelationships between program modules for implementing a generic beam- 
former for automatically designing a set of optimized beams for microphone arrays of arbitrary geometry. It should be 
noted that any boxes and interconnections between boxes that are represented by broken or dashed lines in FIG. 2 
represent alternate embodiments of the generic beamformer described herein, and that any or all of these alternate 

30 embodiments, as described below, may be used in combination with other alternate embodiments that are described 
throughout this document. 

[0081] In general, the generic beamformer operates to design optimized beams for microphone or other sensor 
arrays of known geometry and operational characteristics. Further, these beams are optimized for the local environ- 
ment. In other words, beam optimization is automatically adapted to array geometry, array operational characteristics, 

35 and workspace environment (including the effects of ambient or isotropic noise within the area surrounding the micro- 
phone array, as well as instrumental noise of the microphone array) as a function of signal frequency. 
[0082] Operation of the generic beamformer begins by using each of a plurality of sensors forming a sensor array 
200, such as a microphone array, to monitor noise levels (ambient or isotropic, point source, and instrumental) within 
the local environment around the sensor array. The monitored noise from each sensor, M, in the sensor array 200 is 

40 then provided as an input, x m (n), to a signal input module 205 as a function of time. 

[0083] The next step involves computing one or more noise models based on the measured noise levels in the local 
environment around the sensor array 200. However, in one embodiment, a frequency-domain decomposition module 
21 0 is first used to transform the input signal frames from the time domain to the frequency domain. It should be noted 
that the beamforming operations described herein can be performed using filters that operate either in the time domain 

45 or in the frequency domain. However, for reduced computational complexity, easier integration with other audio process- 
ing elements, and additional flexibility, it is typically better to perform signal processing in the frequency domain. 
[0084] There are many possible frequency-domain signal processing tools that may be used, including, for example, 
discrete Fourier transforms, usually implemented via the fast Fourier transform (FFT). Further, one embodiment of the 
generic beamformer provides frequency-domain processing using the modulated complex lapped transform (MCLT). 

50 Note that the following discussion will focus only on the use of MCLPs rather than describing the use of time-domain 
processing or the use of other frequency-domain techniques such as the FFT. However, it should be appreciated by 
those skilled in the art that the techniques described with respect to the use of the MCLT are easily adaptable to other 
frequency-domain or time-domain processing techniques, and that the generic beamformer described herein is not 
intended to be limited to the use of MCLT processing. 

55 [0085] Therefore, assuming the use of MCLT signal transforms, the frequency-domain decomposition module 21 0 
transforms the input signal frames (representing inputs from each sensor in the array) from the time domain to the 
frequency domain to produce N MCLT coefficients, X^N) for every sensor input, x^n). A noise model computation 
module 215 then computes conventional noise models representing the noise of the local environment around the 
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sensor array 200 by using any of a number of well known noise modeling techniques. However, it should be noted that 
computation of the noise models can be skipped for signal certain frames, If desired. 

[0086] In general, several types of noise models are considered here, including, ambient or isotropic noise within 
the area surrounding the sensor array 200, instrumental noise of the sensor array circuitry, and point noise sources. 

5 Because such noise modeling techniques are well known to those skilled in the art, they will not be described in detail 
herein. Once the noise model computation module 215 has computed the noise models from the input signals, these 
noise models are then provided to a weight computation module 220. In one embodiment, computational overhead is 
reduced by pre-computing the noise models off-line and using those fixed modules; for example a simple assumption 
of isotropic noises (equal energy from any direction and a particular frequency spectral shape). 

w [0087] In addition to the noise models, the weight computation module 220 also receives sensor array parametric 
information 230 which defines geometry and operational characteristics (including directivity patterns) of the sensor 
array 200. For example, when considering a microphone array, the parametric information provided to the generic 
beamformer defines an array of M sensors (microphones), each sensor having a known position vector and directivity 
pattern. As is known to those skilled in the art, the directivity pattern is a complex function, giving the sensitivity and 

is the phase shift, introduced by the microphone for sounds coming from certain locations. 

[0088] Note that there is no requirement for the microphone array to use microphones of the same type or directivity, 
so long as the position and directivity of each microphone is known. Further, as noted above, in one embodiment, this 
sensor array parametric information 230 is provided in a device description file, or a device driver, or the like. Also as 
noted above, in a related embodiment, this parametric information is maintained within the microphone array itself, 

20 and is automatically reported to an external computing device which then operates the generic beamformer in the 
manner described herein. 

[0089] Further, in addition to the noise models and sensor array parametric information 230, the weight computation 
module 220 also receives an input of "target beam shapes" and corresponding "target weight functions" from a target 
beam shape definition module 230. The target beam shape and target weight functions are automatically provided by 

25 a target beam shape definition module 225. In general, as noted above, the target beam shape definition module 230 
defines a set of "target beam shapes" as a function of some desired target beam width focus area around each of a 
number of target focus points. As noted above, defining the optimal target beam shape is best approached as an 
iterative process by producing target beam shapes, and corresponding target weight functions across some desired 
range of target beam widths (i.e., 2-degrees, 5-degrees, 10-degrees, etc.) for each frequency or frequency band of 

30 interest. 

[0090] The number of target focus points used for beamforming computations should generally be larger than the 
number of sensors in the sensor array 200, and in fact, larger numbers tend to provide increased beamforming reso- 
lution. In particular, the number of target focus points L, is chosen to be larger than the number of sensors, M. These 
target focus points are then equally spread in the workspace around the sensor array for beamforming computations. 

35 For example, in a tested embodiment 500 target focus points, L, were selected for a circular microphone array with 8 
microphones, M. These target focus points are then individually evaluated to determine whether they are within the 
target beam width focus area, within a "transition area" around the target beam width focus area, or outside of the 
target beam width focus area and outside the transition area. Corresponding gains provided by the target weight func- 
tions are then applied to each focus point depending upon its position with respect to the beam currently being analyzed. 

40 [0091] In particular, the aforementioned target weight functions are defined as a set of three weighting parameters, 
Vpass* ^Trans* and v stop wnicn correspond to whether the target focus point is within the target beam shape (Vp^J, 
within a "transition area" around the target focus point (V Tmns ), or completely outside the target beam shape and tran- 
sition area ( Note that the transition area is defined by some delta around the perimeter of the target beam shape. 
For example, in a tested embodiment, a delta of three times the target beam width was used to define the transition 

45 area. Thus, assuming a ±10-degree target beam width around the focus point, and assuming a delta of three times 
the target beam width, the transition area would begin at ±1 0-degrees from the target point and extend to ±40-degrees 
from the target point. In this example, everything outside of ±40-degrees around the target point is then in the stop 
area (V stop ). The target weight functions then provide a gain for weighting each target point depending upon where 
those points are relative to a particular target beam. 

so [0092] At this point, the weight computation module 220 has been provided with the target beam shapes, the target 
weight function, the set of target points, the computed noise models, and the directivity patterns of the microphones 
in the microphone array. Given this information, the weight computation module 220 then computes a set of weights 
for each microphone that will fit each real beam shape (using the known directivity patterns of each microphone in the 
array as the real beam shapes) into the current target beam shape for each target point for a current MCLT frequency 

55 subband. Note that as described below in Section 3, this set of weights is optimized by using an error minimization 
technique to choose weights that will minimize the total noise energy for the current MCLT frequency subband. 
[0093] A weight normalization module 235 then normalizes the optimized set of weights for each target beam shape 
to ensure a unit gain and a zero phase shift for any signals originating from the target point corresponding to each 
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target beam shape. 

[0094] The steps described above are then repeated for each of a range of target beam shapes. In other words, the 
steps described above for generating a set of optimized normalized weights for a particular target beam shape are 
repeated throughout a desired range of beam angles using any desired step size. For example, given a step size of 

s 5-degrees, a minimum angle of 10-degrees, and a maximum angle of 60 degrees, optimized normalized weights will 
be computed for each target shape ranging from 10-degrees to 60-degrees in 5-degree increments. As a result, the 
stored target beams and weights 240 will include optimized normalized weights and beam shapes throughout the 
desired range of target beam shapes for each target point for the current MCLT frequency subband. 
[0095] A total noise energy comparison module 245 then computes a total noise energy by performing a simple one- 

10 dimensional search through the stored target beams and weights 240 to identify the beam shape (i.e., the beam angle) 
and corresponding weights that provide the lowest total noise energy around each target point at the current MCLT 
subband. These beam shapes and corresponding weights are then output by an optimized beam and weight matrix 
module 250 as an input to an optimal beam and weight matrix 255 which corresponds to the current MCLT subband. 
[0096] The full optimal beam and weight matrix 255 is then populated by repeating the steps described above for 

is each MCLT subband. in particular, for every MCLT subband, the generic beamformer separately generates a set of 
optimized normalized weights for each target beam shape throughout the desired range of beam angles. As described 
above, the generic beamformer then searches these stored target beam shapes and weights to identify the beam 
shapes and corresponding weights that provide the lowest total noise energy around each target point for each MCLT 
subband, with the beam shapes and corresponding weights then being stored to the optimal beam and weight matrix 

20 255, as described above. 

[0097] Note that except in the case of ideally uniform sensors, such as omni-directional microphones, each sensor 
in the sensor array 200 may exhibit differences in directivity. Further, sensors of different types, and thus of different 
directivity, may be included in the same sensor array 200. Therefore, optimal beam shapes (i.e., those beam shapes 
exhibiting the lowest total noise energy) defined in the optimal beam and weight matrix 255 should be recomputed to 

25 accommodate for sensors of different directivity patterns. 

3.0 Operational Overview: 

[0098] The above-described program modules are employed for implementing the generic beamformer described 
so herein. As described above, the generic beamformer system and method automatically defines a set of optimal beams 
as a function of target point and frequency in the workspace around a sensor array and with respect to local noise 
conditions around the sensor array. The following sections provide a detailed operational discussion of exemplary 
methods for implementing the aforementioned program modules. Note that the terms locus point," "target point," and 
"target focus point" are used interchangeably throughout the following discussion. 

35 

3.1 Initial Considerations: 

[0099] The following discussion is directed to the use of the generic beamformer for defining a set of optimized beams 
for a microphone array of arbitrary, but known, geometry and operational characteristics. However, as noted above, 

40 the generic beamformer described herein is easily adaptable for use with other types of sensor arrays. 

[0100] In addition, the generic beamformer described herein may be adapted for use with filters that operate either 
in the time domain or in the frequency domain. However, as noted above, performing the beamfonming processing in 
the frequency domain provides for reduced computational complexity, easier integration with other audio processing 
elements, and additional flexibility. 

45 [0101] in one embodiment, the generic beamformer uses the modulated complex lapped transform (MCLT) in beam 
design because of the advantages of the MCLT for integration with other audio processing components, such as audio 
compression modules. However, as noted above, the techniques described herein are easily adaptable for use with 
other frequency-domain decompositions, such as the FFT or FFT-based filter banks, for example. 

so 3.1.1 Sensor Array Geometry and Characteristics: 

[01 02] As noted above, the generic beamformer is capable of providing optimized beam design for microphone arrays 
of any known geometry and operational characteristics. In particular, consider an array of M microphones with a known 
positions vector p. The microphones in the array will sample the signal field in the workspace around the array at 
55 locations P m = (x^ z m ) : m = 0, 1 ,— , M- 1 . This sampling yields a set of signals that are denotes by the signal vector 
*(Cph 

[0103] Further, each microphone m has known directivity pattern, U^c), where f is the frequency and c={<p,8,p} 
represents the coordinates of a sound source in a radial coordinate system. A similar notation will be used to represent 
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those same coordinates in a rectangular coordinate system, in this case, c = {x, y, z). As is known to those skilled in 
the art, the directivity pattern of a microphone is a complex function which provides the sensitivity and the phase shift 
introduced by the microphone for sounds coming from certain locations or directions. For an ideal omni-directional 
microphone, U m {f t c) = constant. However, as noted above, the microphone array can use microphones of different 
5 type and directivity patterns without loss of generality of the generic beamformer. 



3.1 .2 Signal Definitions: 



[0104] As is known to those skilled in the art, a sound signal originating at a particular location, c, relative to a 
10 microphone array is affected by a number of factors. For example, given a sound signal, S(f), originating at point c, 
the signal actually captured by each microphone can be defined by Equation (1), as illustrated below: 



X m (/, p m ) = D m (f t c)A(f) m U m (f t c)S(f) Equation (1) 

15 

where the first member, D^c), as defined by Equation (2) below, represents the phase shift and the signal decay due 
to the distance from point c to the microphone. Note that any signal decay due to energy losses in the air is omitted 
as it is significantly lower for working distances typically involved with microphone arrays. However, such losses may 
be more significant when greater distances are involved, or when other sensor types, carrying media (i.e., water, or 
20 other fluids) or signal types are involved. 



25 



The second member of Equation (1), A(f) mt is the frequency response of the microphone array preamplifier/ADC cir- 
cuitry for each microphone, m. The third member of Equation (1), U^c), accounts for microphone directivity relative 
to point c. Finally, as noted above, the fourth member of Equation (1), S(f), is the actual signal itself. 



30 3.1.3 Noise Models: 



[0105] Given the captured signal, X m (1 t p^ t the first task is to compute noise models for modeling various types of 
noise within the local environment of the microphone array. The noise models described herein distinguish three types 
of noise: isotropic ambient nose, instrumental noise and point noise sources. Both time and frequency-domain modeling 
55 of noise sources are well known to those skilled in the art. Consequently, the types of noise models considered will 
only be generally described below. 

[0106] in particular, the isotropic ambient noise, having a spectrum denoted by the term N A (f), is assumed to be 
equally spread throughout the working volume or workspace around the microphone array. This isotropic ambient 
noise, N A (f), is correlated in all channels and captured by the microphone array according to Equation (1). In a tested 

40 embodiment, the noise model N A (f) was obtained by direct sampling and averaging of noise in normal conditions, i.e., 
ambient noise in an office or conference room where the microphone array was to be used. 
[0107] Further, the instrumental noise, having a spectrum denoted by the term N f (0, represents electrical circuit 
noise from the microphone, preamplifier, and ADC (analog/digital conversion) circuitry. The instrumental noise, N/f}, 
is uncorrelated in all channels and typically has close to a white noise spectrum. In a tested embodiment, the noise 

45 model A///) was obtained by direct sampling and averaging of the microphones in the array in an "ideal room" without 
noise and reverberation (so that noises would come only from the circuitry of the microphones and preamplifiers). 
[01 08] The third type of noise comes from distinct point sources that are considered to represent noise. For example, 
point noise sources may include sounds such as, for example, a computer fan, a second speaker that should be 
suppressed, etc. 

50 

3.1 .4 Canonical Form of the Generic Beamformer: 



[0109] As should be clear from the preceding discussion, the beam design operations described herein operate in 
a digital domain rather than directly on the analog signals received directly by the microphone array. Therefore, any 
audio signals captured by the microphone array are first digitized using conventional A/D conversion techniques. To 
avoid unnecessary aliasing effects, the audio signal is preferably processed into frames longer than two times the 
period of the lowest frequency in the MCLT work band. 
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[01 10] Given this digital signal, actual use of the beam design information created by the generic beamformer oper- 
ations described herein is straightforward. In particular, the use of the designed beams to produce an audio output for 
a particular target point based on the total input of the microphone array can be generally described as a combination 
of the weighted sums of the input audio frames captured by the microphone array. Specifically, the output of a particular 
5 beam designed by the beamformer can be represented by Equation (3): 



Y(f) = 5? W m {f)X m {f) Equation (3) 

m=0 



where W m (/) is the weights matrix, W, for each sensor for the target point of interest, and Y(f) is the beamformer output 
15 representing the optimal solution for capturing an audio signal at that target point using the total microphone array 

input. As described above, the set of vectors W m (/) is an A/x/W matrix, where N is the number of MCLT frequency bins 

in the audio frame and M is the number of microphones. Consequently, as illustrated by Equation (3), this canonical 

form of the beamformer guarantees linear processing and absence of non-linear distortions in the output signal Y(f) . 

A block diagram of this canonical beamformer is provided in FIG. 3. 
20 [01 1 1 ] For each set of weights, W(f), there is a corresponding beam shape function, B(f,c), that provides the directivity 

of the beamformer. Specifically, the beam shape function, B(f,c) t represents the microphone array complex-valued 

gain as function of the position of the sound source, and is given by Equation (4): 



25 M-l 

B(f,c) = £ W m (f)D m (f>c)A(f) m U m (f,c) Equation (4) 

m=0 



30 [01 12] It should be appreciated by those skilled in the art, that the general diagram of FIG. 3 can easily be expanded 
to be adapted for more complicated systems. For example, the beams designed by the generic beamformer can be 
used in a number of systems, including, for example, sound source localization (SSL) systems, acoustic echo cancel- 
lation (AEC) systems, directional filtering systems, selective signal capture systems, etc. Further, it should also be 
clear that any such systems may be combined, as desired. 

35 

3.1 .5 Beamformer Parameters: 

[01 13] As is well known to those skilled in the art, one of the purposes of using microphone arrays is to improve the 
signal to noise ratio (SNR) for signals originating from particular points in space, or from particular directions, by taking 
40 advantage of the directional capabilities (i.e., the "directivity") of such arrays. By examining the characteristics of various 
types of noise, and then automatically compensating for such noise, the generic beamformer provides further improve- 
ments in the SNR for captured audio signals. As noted above, three types of noise are considered by the generic 
beamformer. Specifically, isotropic ambient noise, instrumental noise, and point source noise are considered. 

45 3.1 .5.1 Beamformer Noise Considerations: 

[0114] The ambient noise gain, G^f), is modeled as a function of the volume of the total microphone array beam 
within a particular workspace. This noise model is illustrated by Equation (5) which simply shows that the gain for the 
ambient noise, G AN (r), is computed over the entire volume of the combined beam represented by the array as a whole: 

50 

G A N(f) = 7 jjjB(Ac)dc Equation (5) 

v 

55 

where y is the microphone array work volume, i.e., the set of all coordinates c. 

[0115] The instrumental, or non-correlated, noise gain, G /A/ (/). of the microphone array and preamplifiers for any 
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25 



30 



40 



particular target point is modeled simply as a sum of the gains resulting from the weights assigned to the microphones 
in the array with respect to that target point. In particular, as illustrated by Equation (6), the non-correlated noise gain, 
G«y/(0» from the microphones and the preamplifiers is given by: 



G lN {f) = v£w m (f) 2 Ration (6) 



[0116] Finally, gains for point noise sources are given simply by the gain associated with the beam shape for any 
particular beam. In other words, the gain for a noise source at point c is simply given by the gain for the beam shape 

Btfc). 

[01 1 7] In view of the gains associated with the various types of noise, a total noise energy in the beamf ormer output 
is is given by Equation (7): 



*> En = J ^G AN (f)N AN (f)) 2 HG lN (f)Ni (f)) 2 df Equation (7) 



3.1 .5.2 Beamformer Directivity Considerations: 

[01 18] In addition to considering the effects of noise, the generic beamformer also characterizes the directivity of the 
microphone array resulting from the beam designs of the generic beamformer. In particular, the directivity index Dl, of 
the microphone array can be characterized by Equations (8) through (10), as illustrated below: 

P(f,<p,G) = \B(f t c)\ 2 , p » p 0 = const Equation (8) 



D= ) df Equatjon(9) 

f= °j-\dO ]d<p P(f,q>,d) 



4 *o o 



Dl = 1 0log 10 D Equation (1 0) 



where P{f,y,Q) is called a "power pattern," p 0 is the average distance (depth) of the work volume, and (<p 7 ,e r ; is the 
45 steering direction. 

3.2 Problem Definition and Constraints: 



[0119] In general, the two main problems faced by the generic beamformer in designing optimal beams for the mi- 
50 crophone array are: 

1 . Calculating the aforementioned weights matrix, W, for any desired focus point, c T , as used in the beamformer 
illustrated by Equation (3); and 

55 2. Providing maximal noise suppression, i.e., minimizing the total noise energy (see Equation (7), for example) in 

the output signal under the constraints of unit gain and zero phase shift in the focus point for the work frequency 
band. These constraints are illustrated by Equation (11), as follows: 
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\B(f,c T )\=\ 



where F B£G and t END represent the boundaries of the work frequency band. 

[0120] These constraints, unit gain and zero phase shift in the focus or target point, are applied for an area around 
the focus point, called focus width. Given the aforementioned noise models, the generic solution of the problems noted 
w above are similar to a typical minimization problem with constraints which may be solved using methods for mathe- 
matical multidimensional optimization (i.e., simplex, gradient, etc.). Unfortunately, due to the high dimensionality of the 
weight matrix W (2M real numbers per frequency band, for a total of Nx2M numbers), a multimodal hypersurf ace, and 
because the functions are nonlinear, finding the optimal weights as points in the multimodal hypersurface is very com- 
putationally expensive, as it typically requires multiple checks for local minima. 
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3.3 Low Dimension Error Minimization Solution far Weight Matrix, W: 



[0121] While there are several conventional methods for attempting to solve the multimodal hypersurface problem 
outlined above, such methods are typically much too slow to be useful in beamforming systems where a fast response 
20 js desired for beamforming operations. Therefore, rather than directly attempting to solve this problem, the direct mul- 
tidimensional optimization of the function defined by Equation (7) under the constraints of Equation (11) is addressed 
by using a least-squares, or other error minimization technique, error pattern synthesis followed by a single dimensional 
search towards the focus width for each target or focus point around the microphone array. 

[0122] Considering the two constraints of Equation (11), it should be clear that there are two contradicting processes. 

25 [0123] In particular, given a narrow focus area, the first constraint of Equation (1 1 ), unit gain at the focus point, tends 
to force the ambient noise energy illustrated in Equation (7) to decrease as a result of increased directivity resulting 
from using a narrow focus area. Conversely, given a narrow focus area, the non-correlated noise energy component 
of Equation (7) will tend to increase due to that fact that the solution for better directivity tries to exploit smaller and 
smaller phase differences between the signals from microphones, thereby boosting the non-correlated noise within 

30 the circuitry of the microphone array. 

[0124] On the other hand, when the target focus area is larger there is more ambient noise energy within that area, 
simply by virtue of the larger beam width. However, the non-correlated noise energy goes down, since the phase 
differences between the signals from the microphone become less important, and thus the noise effects of the micro- 
phone array circuitry has a smaller effect. 

35 [0125] Optimization of these contradicting processes results in a weight matrix solution for the focus area width 
around any given focus or target point where the total noise energy illustrated by Equation (7) is a minimum. The 
process for obtaining this optimum solution is referred to herein as "pattern synthesis." In general, this pattern synthesis 
solution finds the weights for the weights matrix of the optimum beam shape which minimizes the error (using the 
aforementioned least squares or other error minimization technique) for a given target beam shape. Consequently, the 

40 solution for the weight matrix is achieved using conventional numerical methods for solving a linear system of equations. 
Such numerical methods are significantly faster to achieve than conventional multidimensional optimization methods. 

3.3.1 Define Set of Target Beam Shapes: 

45 [0126] In view of the error minimization techniques described above, defining the target beam shapes is a more 
manageable problem. In particular, the target beam shapes are basically a function of one parameter - the target focus 
area width. As noted above, any function with a maximum of one, and which decays to zero can be used to define the 
target beam shape (this function provides gain within the target beam, i.e., a gain of one at the focus point which then 
decays to zero at the beam boundaries). However, abrupt functions, such as rectangular functions, which define a 

so rectangular target area, tend to cause ripples in the beam shape, thereby decreasing overall performance of the generic 
beamformer. Therefore, better results are achieved by using target shape functions that smoothly transition from one 
to zero. 

[01 27] One example of a smoothly decaying function that was found to produce good results in a tested embodiment 
is a conventional cosine-shaped function, as illustrated by Equation (12), as follows: 

55 
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r (A ^^^) = co S (^^)cos(£^)cos[£^) Equation (12) 

5 

where (p 7> (p 7 , G r ) is the target focus point, 8 is the target area size, and k is a scaling factor for modifying the shape 
function. 

[0128] in addition, as noted above, the aforementioned target weight function, V(p, <p, 6), is defined as a set of three 
io weighting parameters, V Tmnsr and which correspond to whether the target focus point is within the target 
beam shape (V Pass ), within a "transition area" around the target focus point (V T/Hns ), or completely outside the target 
beam shape and transition area ( V^^. As discussed in greater detail in Section 2.1 , the target weight functions provide 
a gain for weighting each target point depending upon where those points are relative to a particular target beam, with 
the purpose of such weighting being to minimize the effects of signals originating from points outside the main beam 
15 on beamformer computations. 

3.3.2 Pattern Synthesis: 

[01 29] Once the target beam shape and the target weight functions are defined, it is a simple matter to identify a set 
20 of weights that fit the real beam shape (based on microphone directivity patterns) into the target function by satisfying 
the least square requirement (or other error minimization technique). 

[0130] In particular, the first step is to choose L points, with L > M, equally spread in the work space. Then, for a 
given frequency ( the beam shapes T (see Equation (12)) for given focus area width 8 can be defined as the complex 
product of the target weight functions, V, the number of microphones in the array, M, the phase shift and signal decay 
25 D (see Equation (2)), the microphone directivity responses U, and the weights matrix or "weights vector" W. This product 
can be represented by the complex equation illustrated by Equation (13): 



Tixl = V 1xL D MxL U MxL W 1xM Equation (13) 

30 

The solution to this complex equation (i.e., solving for the optimal weights, W) is then identified by finding the minimum 
mean-square error (MMSE) solution (or the minimum using other conventional error minimization techniques) for the 
weights vector W. Note that this weights vector W is denoted below by w. 

35 3.3.3 Normalization of Weights: 

[0131] The weight solutions identified in the pattern synthesis process described in Section 3.3.2 fits the actual 
directivity pattern of each microphones in the array to the desired beam shape 7. However, as noted above, these 
weights do not yet satisfy the constraints in Equation (11 ). Therefore, to address this issue, the weights are normalized 
40 to force a unit gain and zero phase shift for signals originating from the focus point c T . This normalization is illustrated 
by Equation (14), as follows: 

A 

W=pT7^1 Equation (14) 

45 B V' C J 

where W represents the optimized normalized weights under the constraints of Equation (11). 
3.3.4 Optimization of Beam Width: 

50 

[0132] As discussed above, for each frequency, the processes described above in sections 3.3.1 through 3.3.3 for 
identifying and normalizing weights that provide the minimum noise energy in the output signal are then repeated for 
each of a range of target beam shapes, using any desired step size. In particular, these processes are repeated through- 
out a range, [S M/W , 8 MAX ], where 8 represents the target area width around each particular target focus point. In other 
55 words, the repeat the discussion provided above, the processes described above for generating a set of optimized 
normalized weights, i.e., weights vector ff(f), for a particular target beam shape are repeated throughout a desired 
range of beam angles using any desired step size for each target point for the current MCLT frequency subband. The 
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resulting weights vector w(1) is the "pseudo-optimal" solution for a given frequency f. 

3.3.5 Calculation for the Whole Frequency Band: 

5 [0133] To obtain the full weights matrix w for a particular target focus point, the processes described in Section 3.3.1 
through 3.3.4 are then simply repeated for each MCLT frequency subband in the frequency range being processed by 
the microphone array. 

3.3.6 Calculation of the Beams Set: 

10 

[0134] After completing the processes described in Sections 3.3.1 through 3.3.5, the weights matrix w, then repre- 
sents an Nx M matrix of weights for a single beam for a particular focus point c T Consequently, the processes described 
above in Sections 3.3. 1 through 3.3.5 are repeated Ktimes for Kbeams, with the beams being evenly placed throughout 
the workspace. The resulting A/xMxKthree-dimensional weight matrix specifies the full beam design produced by the 
is generic beamformerfor the microphone array in its current local environment given the current noise conditions of that 
local environment. 

4.0 Implementation 

20 [0135] In one embodiment, the beamforming processes described above in Section 3 for designing optimal beams 
for a particular sensor array given local noise conditions is implemented as two separate parts: an off-line design 
program that computes the aforementioned weight matrix, and a run-time microphone array signal processing engine 
that uses those weights according to the diagram in FIG. 3. One reason for computing the weights offline is that it is 
substantial ly more computationally expensive to compute the optimal weights than it is to use them in the signal process- 
es ing operation illustrated by FIG. 3. 

[01 36] However, given the speed of conventional computers, including, for example, conventional PC-type comput- 
ers, real-time, or near real-time computations of the weights matrix is possible. Consequently, in another embodiment, 
the weights matrix is computed in an ongoing basis, in as near to real-time as the available computer processing power 
allows. As a result, the beams designed by the generic beamformer are continuously and automatically adapting to 
30 changes in the ambient noise levels in the local environment. 

[0137] The processes described above with respect to FIG. 2 and FIG. 3, and in further view of the detailed description 
provided in Sections 2 and 3 are illustrated by the general operational flow diagram of FIG. 5. In particular, FIG. 5 
provides an exemplary operational flow diagram which illustrates operation of the generic beamformer. It should be 
noted that any boxes and interconnections between boxes that are represented by broken or dashed lines in FIG. 5 
35 represent alternate embodiments of the generic beamformer described herein, and that any or all of these alternate 
embodiments, as described below, may be used in combination with other alternate embodiments that are described 
throughout this document. 

[0138] In general, as illustrated by FIG. 5, beamforming operations begin by monitoring input signals (Box 505) from 
a microphone array 500 over some period of time sufficient to generate noise models from the array input. In general, 
40 as is known to those skilled in the art, noise models can be computed based on relatively short samples of an input 
signal. Further, as noted above, in one embodiment, the microphone array 500 is monitored continuously, or at user 
designated times or intervals, so that noise models may be computed and updated in real-time or in near-real time for 
use in designing optimal beams for the microphone array which adapt to the local noise environment as a function of 
time. 

45 [01 39] Once the input signal has been received, conventional A/D conversion techniques 51 0 are used to construct 
digital signal frames from the incoming audio signals. As noted above, the length of such frames should typically be 
at least two or more times the period of the lowest frequency in the MCLT work band in order to reduce or minimize 
aliasing effects. The digital audio frames are then decomposed into MCLT coefficients 515. In a tested embodiment, 
the use of 320 MCLT frequency bands was found to provide good results when designing beams for a typical circular 

so microphone array in a typical conference room type environment. 

[0140] At this point, since the decomposed audio signal is represented as a frequency-domain signal by the MCLT 
coefficients, it is rather simple to apply any desired frequency domain processing, such as, for example filtering at 
some desired frequency or frequency range. For example, where ft is desired to exclude ail but some window of fre- 
quency ranges from the noise models, a bandpass type filter may be applied at this step. Similarly, other filtering effects, 

55 including, for example high-pass, low-bass, multi-band filters, notch filters, etc, may also be applied, either individually, 
or in combination. Therefore, in one embodiment, preprocessing 520 of the input audio frames is performed prior to 
generating the noise models from the audio frames. 

[0141] These noise models are then generated 525, whether or not any preprocessing has been performed, using 
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conventional noise modeling techniques. For example, isotropic ambient noise is assumed to be equally spread 
throughout the working volume or workspace around the microphone array. Therefore, the isotropic ambient noise is 
modeled by direct sampling and averaging of noise in normal conditions in the location where the array is to be used. 
Similarly, instrumental noise is modeled by direct sampling and averaging of the microphones in the array in an "ideal 
5 room" without noise and reverberation (so that noises would come only from the circuitry of the microphones and 
preamplifiers). 

[0142] Once the noise models have been generated 525, the next step is to define a number of variables (Box 530) 
to be used in the beamforming design. In particular, these variables include: 1) the target beam shapes, based on 
some desired decay function, as described above; 2) target focus points, spread around the array; 3) target weight 

io functions, for weighting target focus points depending upon whether they are in a particular target beam, within a 
transition area around that beam, or outside the beam and transition area; 4) minimum and maximum desired beam 
shape angles; and 5) a beam step size for incrementing target beam width during the search for the optimum beam 
shape. Note that all of these variables may be predefined for a particular array and then simply read back for use in 
beam design. Alternately, one or more of these variables are user adjustable to provide for more user control over the 

is beam design process. 

[0143] Counters for tracking the current target beam shape angle (i.e., the current target beam width), current MCLT 
subband, and current target beam at point c^fc) are then initialized (Box 535) prior to beginning the beam design 
process represented by the steps illustrated in Box 540 through Box 585. 

[0144] In particular, given the noise models and the aforementioned variables, optimal beam design begins by first 

20 computing weights 540 for the current beam width at the current MCLT subband for each microphone and target focus 
point given the directivity of each microphone. As noted above, the microphone parametric information 230 is either 
maintained in some sort of table or database, or in one embodiment, it is automatically stored in, and reported by the 
microphone array itself, e.g., the "Self -Descriptive Microphone Array" described above. These computed weights are 
then normalized 550 to ensure unit gain and zero phase shift at the corresponding target focus point. The normalized 

25 weights are then stored along with the corresponding beam shape 240. 

[0145] Next, a determination 555 is made as to whether the current beam shape angle is greater than or equal to 
the specified maximum angle from step 530. If the current beam angle is less than the maximum beam angle specified 
in step 530, then the beam angle is incremented by the aforementioned beam angle step size (Box 560). A new set 
of weights are then computed 540, normalized 550, and stored 240 based on the new target beam width. These steps 

30 (540, 550, 240, and 555) then repeat until the target beam width is greater than or equal to the maximum angle 555. 
[0146] At this point, the stored target beams and corresponding weights are searched to select the optimal beam 
width (Box 565) for the current MCLT band for the current target beam at point c^rt). This optimal beam width and 
corresponding weights vector are then stored to the optimal beam and weight matrix 255 for the current MCLT subband. 
A determination (Box 570) is then made as to whether the current MCLT subband, e.g., MCLT subband (i), is the 

35 maximum MCLT subband. If it is not, then the MCLT subband identifier, (i), is incremented to point to the next MCLT 
subband, and the current beam width is reset to the minimum angle (Box 575). 

[0147] The steps described above for computing the optimal beam and weight matrix entry for the current MCLT 
subband (540, 550, 240, 555, 560, 565, 255, 570, and 575) are then repeated by the new current MCLT subband until 
the current MCLT subband is equal to the maximum MCLT subband (Box 570). Once the current MCLT subband is 
40 equal to the maximum MCLT subband (Box 570), then the optimal beam and weight matrix will have been completely 
populated across each MCLT subband for the current target beam at point c-rfk). 

[0148] However, it is typically desired to provide for more than a single beam for a microphone array. Therefore, as 
illustrated by steps 580 and 585, the steps described above for populating the optimal beam and weight matrix each 
MCLT subband for the current target beam at point c^/c) are repeated K times for K beams, with the beams usually 
45 being evenly placed throughout the workspace. The resulting NxMxK three-dimensional weight matrix 255 specifies 
the full beam design produced by the generic beamformer for the microphone array in its current local environment 
given the current noise conditions of that local environment. 

[01 49] The foregoing description of the generic beamformer for designing a set of optimized beams for microphone 
arrays of arbitrary geometry and microphone directivity has been presented for the purposes of illustration and de- 
50 scription. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications 
and variations are possible in light of the above teaching. Further, it should be noted that any or all of the aforementioned 
alternate embodiments may be used in any combination desired to form additional hybrid embodiments of the generic 
beamformer. It is intended that the scope of the invention be limited not by this detailed description, but rather by the 
claims appended hereto. 

55 
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Claims 

1. A method for real-time design of beam sets for a microphone array from a set of pre-computed noise models, 
comprising using a computing device to: 

5 

compute a set of complex-valued gains for each subband of a frequency-domain decomposition of microphone 
array signal inputs for each of a plurality of beam widths within a range of beam widths, said sets of complex- 
valued gains being computed from the pre-computed noise models in combination with known geometry and 
directivity of microphones comprising the microphone array; 
10 search the sets of complex-valued gains to identify a single set of complex-valued gains for each frequency- 

domain subband and for each of a plurality of target focus points around the microphone array; and 

wherein each said set of complex-valued gains is individually selected as the set of complex-valued gains 
having a lowest total noise energy relative to corresponding sets of complex-valued gains for each frequency- 
's domain subband for each target focus point around the microphone array, and wherein each selected set of com- 
plex-valued gains is then provided as an entry in said beam set for the microphone array. 

2. The method of claim 1 wherein the frequency-domain decomposition is a Modulated Complex Lapped Transform 
(MCLT). 

20 

3. The method of claim 1 wherein the frequency-domain decomposition is a Fast Fourier Transform (FFT). 

4. The method of claim 1 wherein the pre-computed noise models include at least one of ambient noise models, 
instrumental noise models, and point source noise models. 

25 

5. The method of claim 4 wherein the ambient noise models are computed by direct sampling and averaging of 
isotropic noise in a workspace around the microphone array. 

6. The method of claim 4 wherein the instrumental noise models are computed by direct sampling and averaging of 
30 the output of the microphones in the microphone array in a workspace without noise and reverberation, so that 

only those noises originating from the circuitry of the microphone array is sampled. 

7. The method of claim 1 wherein the total noise energy is computed as a function of the pre-computed noise models 
and the beam widths in combination with the corresponding sets of complex-valued gains. 

35 

8. The method of claim 1 wherein at least one member of the set of pre-computed noise models is recomputed in 
real-time in response to changes in noise levels around the microphone array. 

9. The method of claim 1 wherein the sets of complex-valued gains are normalized to ensure unit gain and zero 
40 phase shift for signals originating from each target focus point. 

10. The method of claim 1 wherein the range of beam widths is defined by a pre-determined minimum beam width, a 
pre-determined maximum beam width, and a pre-determined beam width step size. 

« 11. The method of claim 1 wherein the range of beam widths is defined by a user adjustable minimum beam width, a 
user adjustable maximum beam width, and a user adjustable beam width step size. 

12. The method of claim 1 wherein the known geometry and directivity of the microphones comprising the microphone 
array are provided from a device description file which defines operational characteristics of the microphone array. 

50 

13. The method of claim 12 wherein the device description file is internal to the microphone array, and wherein the 
known geometry and directivity of the microphones comprising the microphone array are automatically reported 
to the computing device for use in the real-time design of beam sets. 

55 14. The method of claim 1 further comprising a beamforming processor for applying the beam set for real-time process- 
ing of incoming microphone signals from the microphone array. 

15. A system for automatically designing beam sets for a sensor array, comprising: 
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means for monitoring ail sensor signal outputs of a sensor array having a plurality of sensors, each sensor 
having a known geometry and directivity pattern; 

means for generating at least one noise model from the sensor signal outputs; 

means for defining a set of target beam shapes as a function of a set of target beam focus points and a range 
5 of target beam widths, said target beam focus points being spatiaily distributed within a workspace around the 

sensor array; 

means for defining a set of target weight functions to provide a gain for weighting each target focus point 
depending upon the position of each target focus point relative to a particular target beam shape; 
means for computing a set of potential beams by computing a set of normalized weights for fitting the directivity 
10 pattern of each microphone into each target beam shape throughout the range of target beam widths across 

a frequency range of interest for each weighted target focus point; 

means for identifying a set of beams by computing a total noise energy for each potential beam across a 
frequency range of interest, and selecting each potential beam having a lowest total noise energy for each of 
a set of frequency bands across the frequency range of interest. 

15 

1 6. The system of claim 1 5 wherein the normalized weights represent sets of complex-valued gains for each subband 
of a frequency-domain decomposition of sensor array signal inputs. 

17. The system of claim 1 6 wherein the frequency-domain decomposition is a Modulated Complex Lapped Transform 
20 (MCLT). 

18. The system of claim 16 wherein the frequency-domain decomposition is a Fast Fourier Transform (FFT). 

19. The system of claim 15 wherein generating the at least one noise model from the sensor signal outputs comprises 
25 computing at least one of an ambient noise model, an instrumental noise model, and a point source noise model 

through direct sampling and analysis of noise in a workspace around the sensor array. 

20. The system of claim 15 wherein computing the total noise energy for each potential beam across a frequency 
range of interest comprises determining noise energy levels as a function of the at least one noise model and the 

30 normalized weights associated with each potential beam. 

21. The system of claim 15 wherein at least one of the noise models is automatically recomputed in real-time in re- 
sponse to changes in noise levels around the sensor array. 

35 22. The system of claim 1 5 wherein the normalized weights for each potential beam ensure unit gain and zero phase 
shift for signals originating from each corresponding target focus point. 

23. The system of claim 1 5 wherein the range of target beam widths is limited by minimum and maximum beam widths 
in combination with a beam width angle step size for selecting specific target beam widths across the range of 

40 target beam widths. 

24. The system of claim 1 5 wherein the known geometry and directivity of each sensor is automatically provided from 
a device description file residing within the sensor array. 

45 25. The system of claim 1 5 further comprising a beamforming processor for real-time steerable beam-based process- 
ing of sensor array inputs by applying the set of beams to the sensor array inputs for particular target focus points. 

26. A computer-readable medium having computer executable instructions for automatically designing a set of steer- 
able beams for processing output signals of a microphone array, said computer executable instructions comprising: 

50 

computing sets of complex-valued gains for each of a plurality of beams through a range of beam widths for 
each of a plurality of target focus points around the microphone array from a set of parameters, said parameters 
including one or more models of noise of an environment within range of microphones in the microphone array 
and known geometry and directivity patterns of each microphone in the microphone array; 

55 

wherein each beam is automatically selected throughout the range of beam widths using a beam width angle 
step size for selecting specific beam widths across the range of beam widths; 

computing a lowest total noise energy for each set of complex-valued gains for each target focus point for 
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each beam width; and 

identifying the sets of complex-valued gains and corresponding beam width having the lowest total noise 
energy for each target focus point, and selecting each such set as a member of the set of steerable beams for 
processing the output signals of a microphone array. 

5 

27. The computer readable medium of claim 26 wherein the complex-valued gains are normalized to ensure unit gain 
and zero phase shift for signals originating from corresponding target focus points. 

28. The computer readable medium of claim 26 wherein the complex-valued gains are separately computed for each 
10 subband of a frequency-domain decomposition of microphone array input signals. 

29. The computer readable medium of claim 28 wherein the frequency-domain decomposition is any of a Modulated 
Complex Lapped Transform (MCLT)-based decomposition, and a Fast Fourier Transform (FFT)-based decompo- 
sition. 

15 

30. The computer readable medium of claim 26 further comprising a beamforming processor for applying the set of 
steerable beams for processing output signals of the microphone array. 

31 . The computer readable medium of claim 30 wherein the beamforming processor comprises a sound source local- 
20 ization (SSL) system for using the optimized set of steerable beams for localizing audio signal sources within an 

environment around the microphone array. 

32. The computer readable medium of claim 31 wherein the beamforming processor comprises an acoustic echo 
cancellation (AEC) system for using the optimized set of steerable beams for canceling echoes outside of a par- 

25 ticular steered beam. 

33. The computer readable medium of claim 31 wherein the beamforming processor comprises a directional filtering 
system for selectively filtering audio signal sou rces relative to the target focus point of one or more steerable beams. 

30 34. The computer readable medium of claim 31 wherein the beamforming processor comprises a selective signal 
capture system for selectively capturing audio signal sources relative to the target focus point of one or more 
steerable beams. 

35. The computer readable medium of claim 31 wherein the beamforming processor comprises a combination of two 
35 or more of: 

a sound source localization (SSL) system for using the optimized set of steerable beams for localizing audio 
signal sources within an environment around the microphone array; 

an acoustic echo cancellation (AEC) system for using the optimized set of steerable beams for canceling 
40 echoes outside of a particular steered beam; 

a directional filtering system for selectively filtering audio signal sources relative to the target focus point of 
one or more steerable beams; and 

a selective signal capture system for selectively capturing audio signal sources relative to the target focus 
point of one or more steerable beams. 

45 
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